Skip to content

feat(ai-red-teaming): agentic-suite tool (run all attacks on an agent) + scenario docs - #112

Open
rdheekonda wants to merge 2 commits into
mainfrom
feat/atlas-devops-support-scenarios
Open

feat(ai-red-teaming): agentic-suite tool (run all attacks on an agent) + scenario docs#112
rdheekonda wants to merge 2 commits into
mainfrom
feat/atlas-devops-support-scenarios

Conversation

@rdheekonda

@rdheekonda rdheekonda commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes the AI-red-teaming capability able to run all possible attacks against an agent in one shot — the "red team my agent, here are the details" path in the TUI — plus the devops/support scenario docs.

Companion PRs: dreadnode-tiger SDK (run_agentic_suite runner + arg-aware gate) #2181, security-tasks environments #146.

What's included

  • generate_agentic_suite_attack tool (tools/attacks.py) — point it at an agent HTTP endpoint (preset or custom request/response template) with an attacker_model; omit categories to run every OWASP-ASI category. Auto-selects the mapped attacks, family transforms (MCP, multi-agent, reasoning, exfiltration, …), and detection scorers via the SDK's run_agentic_suite. No need to name individual attacks.
  • Workflow generator (scripts/attack_runner.py) — emits a workflow that builds the agent target and calls run_agentic_suite; registered in the runner method table.
  • Agent wiring (agents/ai-red-teaming-agent.md) — added to the decision tree ("run all possible attacks / red team my agent → generate_agentic_suite_attack") and the tool reference, so the TUI agent picks it for broad/complete audits.
  • Docs: generate_atlas_attack's scenario_name now lists devops/support + env-derived names; environments.py lists the new devops-rce-mesh/support-exfil-mesh targets.

Validation

  • Capability test TestScriptGeneration::test_agentic_suite passes: the generator produces a syntactically valid workflow that calls run_agentic_suite( and defines the agent target(.
  • Blocking CI lint (ruff --select E9,F63,F7,F82) clean on the changed files.

Honest status / follow-up

  • The generator is generate-verified (syntax + correct references), matching the bar used for generate_atlas_attack. A live TUI end-to-end run of the full suite lands once the companion SDK (run_agentic_suite) is released and the capability runtime picks it up — I did not fake an end-to-end claim here.

Test plan

  • CI: capability lint + manifest validation green
  • Reviewer: confirm the agent decision tree routes "run all attacks" to the new tool
  • Post-SDK-release: live TUI run of generate_agentic_suite_attack against a mesh

generate_atlas_attack's scenario_name now lists devops (RCE) and support
(exfil) and notes the environment-derived names (devops-rce, support-exfil)
resolve to the shipped argument-aware policy. environments doc lists the new
devops-rce-mesh / support-exfil-mesh targets.
… on an agent

Adds a turnkey capability tool + workflow generator that runs the FULL agentic
suite (every OWASP-ASI category, auto-selecting the mapped attacks, family
transforms, and detection scorers via the SDK's run_agentic_suite) against an
agent HTTP endpoint — the "run all possible attacks / red team my agent" path,
no need to name individual attacks. Wires it into the agent decision tree + tool
list so the TUI agent picks it for broad/complete agent audits. Generate-verified
+ capability test; live TUI run pending SDK release.
@rdheekonda rdheekonda changed the title docs(ai-red-teaming): document devops/support ATLAS scenarios + envs feat(ai-red-teaming): agentic-suite tool (run all attacks on an agent) + scenario docs Aug 5, 2026
@rdheekonda
rdheekonda marked this pull request as ready for review August 6, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant